Multi-modal computer-aided diagnosis systems and methods for prostate cancer

ABSTRACT

Methods and apparatus for computer-aided prostate condition diagnosis are disclosed. An example computer-aided prostate condition diagnosis apparatus includes a memory to store instructions and a processor. The example processor is to execute the instructions to implement at least a prostate assessor, a lesion assessor, and an outcome generator. The example prostate assessor is to evaluate a volume and density of a prostate gland in an image of a patient to determine a prostate-specific antigen level for the prostate gland. The example lesion assessor is to analyze a lesion on the prostate gland in the image. The example outcome generator is to generate an assessment of prostate gland health based on the prostate-specific antigen level and the analysis of the lesion.

CROSS-REFERENCE TO RELATED APPLICATION

This patent arises from U.S. Provisional Patent Application Ser. No.62/590,266, which was filed on Nov. 22, 2017. U.S. Provisional PatentApplication Ser. No. 62/590,266 is hereby incorporated herein byreference in its entirety. Priority to U.S. Provisional PatentApplication Ser. No. 62/590,266 is hereby claimed.

FIELD OF THE DISCLOSURE

This disclosure relates generally to improved medical systems and, moreparticularly, to improved computer-aided diagnosis systems and methodsfor medical image processing.

BACKGROUND

A variety of economy, technological, and administrative hurdleschallenge healthcare facilities, such as hospitals, clinics, doctors'offices, etc., to provide quality care to patients. Economic drivers,less skilled staff, fewer staff, complicated equipment, and emergingaccreditation for controlling and standardizing radiation exposure doseusage across a healthcare enterprise create difficulties for effectivemanagement and use of imaging and information systems for examination,diagnosis, and treatment of patients.

Healthcare provider consolidations create geographically distributedhospital networks in which physical contact with systems is too costly.At the same time, referring physicians want more direct access tosupporting data in reports along with better channels for collaboration.Physicians have more patients, less time, and are inundated with hugeamounts of data, and they are eager for assistance.

Healthcare provider tasks including image processing and analysis, etc.,are time consuming and resource intensive tasks impractical, if notimpossible, for humans to accomplish alone.

BRIEF DESCRIPTION

Certain examples provide a computer-aided prostate condition diagnosisapparatus. The example apparatus includes a memory to store instructionsand a processor. The example processor is to execute the instructions toimplement at least a prostate assessor, a lesion assessor, and anoutcome generator. The example prostate assessor is to evaluate a volumeand density of a prostate gland in an image of a patient to determine aprostate-specific antigen level for the prostate gland. The examplelesion assessor is to analyze a lesion on the prostate gland in theimage. The example outcome generator is to generate an assessment ofprostate gland health based on the prostate-specific antigen level andthe analysis of the lesion.

Certain examples provide a computer-readable storage medium includinginstructions. The instructions, when executed, cause at least oneprocessor to at least: evaluate a volume and density of a prostate glandin an image of a patient to determine a prostate-specific antigen levelfor the prostate gland; analyze a lesion on the prostate gland in theimage; and generate an assessment of prostate gland health based on theprostate-specific antigen level and the analysis of the lesion.

Certain examples provide a method for computer-aided prostate conditiondiagnosis. The example method includes evaluating, with at least oneprocessor, a volume and density of a prostate gland in an image of apatient to determine a prostate-specific antigen level for the prostategland. The example method includes analyzing, with the at least oneprocessor, a lesion on the prostate gland in the image. The examplemethod includes generating, with the at least one processor, anassessment of prostate gland health based on the prostate-specificantigen level and the analysis of the lesion.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an example imaging system to which the methods,apparatus, and articles of manufacture disclosed herein can be applied.

FIG. 1B illustrates an example computer-aided prostate analysis system.

FIG. 2 depicts an example digital twin environment.

FIG. 3 is a representation of an example learning neural network.

FIG. 4 illustrates a particular implementation of the example neuralnetwork as a convolutional neural network.

FIG. 5 is a representation of an example implementation of an imageanalysis convolutional neural network.

FIG. 6A illustrates an example configuration to apply a learning networkto process and/or otherwise evaluate an image.

FIG. 6B illustrates a combination of a plurality of learning networks.

FIG. 7 illustrates example training and deployment phases of a learningnetwork.

FIG. 8 illustrates an example product leveraging a trained networkpackage to provide a deep learning product offering.

FIGS. 9A-9C illustrate various deep learning device configurations.

FIG. 10 illustrates a flow diagram of an example method forcomputer-driven prostate analysis.

FIGS. 11-19C depict example interfaces facilitate prostate analysis andassociated patient diagnosis and treatment.

FIG. 20 is a block diagram of a processor platform structured to executethe example machine readable instructions to implement componentsdisclosed and described herein.

The foregoing summary, as well as the following detailed description ofcertain embodiments of the present invention, will be better understoodwhen read in conjunction with the appended drawings. For the purpose ofillustrating the invention, certain embodiments are shown in thedrawings. It should be understood, however, that the present inventionis not limited to the arrangements and instrumentality shown in theattached drawings. The figures are not scale. Wherever possible, thesame reference numbers will be used throughout the drawings andaccompanying written description to refer to the same or like parts.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying drawings that form a part hereof, and in which is shown byway of illustration specific examples that may be practiced. Theseexamples are described in sufficient detail to enable one skilled in theart to practice the subject matter, and it is to be understood thatother examples may be utilized and that logical, mechanical, electricaland other changes may be made without departing from the scope of thesubject matter of this disclosure. The following detailed descriptionis, therefore, provided to describe an exemplary implementation and notto be taken as limiting on the scope of the subject matter described inthis disclosure. Certain features from different aspects of thefollowing description may be combined to form yet new aspects of thesubject matter discussed below.

When introducing elements of various embodiments of the presentdisclosure, the articles “a,” “an,” and “the” are intended to mean thatthere are one or more of the elements. The terms “first,” “second,” andthe like, do not denote any order, quantity, or importance, but ratherare used to distinguish one element from another. The terms“comprising,” “including,” and “having” are intended to be inclusive andmean that there may be additional elements other than the listedelements. As the terms “connected to,” “coupled to,” etc. are usedherein, one object (e.g., a material, element, structure, member, etc.)can be connected to or coupled to another object regardless of whetherthe one object is directly connected or coupled to the other object orwhether there are one or more intervening objects between the one objectand the other object.

As used herein, the terms “system,” “unit,” “module,” “engine,” etc.,may include a hardware and/or software system that operates to performone or more functions. For example, a module, unit, or system mayinclude a computer processor, controller, and/or other logic-baseddevice that performs operations based on instructions stored on atangible and non-transitory computer readable storage medium, such as acomputer memory. Alternatively, a module, unit, engine, or system mayinclude a hard-wired device that performs operations based on hard-wiredlogic of the device. Various modules, units, engines, and/or systemsshown in the attached figures may represent the hardware that operatesbased on software or hardwired instructions, the software that directshardware to perform the operations, or a combination thereof.

In addition, it should be understood that references to “one embodiment”or “an embodiment” of the present disclosure are not intended to beinterpreted as excluding the existence of additional embodiments thatalso incorporate the recited features.

Overview

Imaging devices (e.g., gamma camera, positron emission tomography (PET)scanner, computed tomography (CT) scanner, X-Ray machine, magneticresonance (MR) imaging machine, ultrasound scanner, etc.) generatemedical images (e.g., native Digital Imaging and Communications inMedicine (DICOM) images) representative of the parts of the body (e.g.,organs, tissues, etc.) to diagnose and/or treat diseases. MR, forexample, is a medical imaging modality that generates images of theinside of a human body without using x-rays or other ionizing radiation.MR uses a main magnet to create a strong, uniform, static magnetic field(e.g., the “main magnetic field”) and gradient coils to produce smalleramplitude, spatially varying magnetic fields when a current is appliedto the gradient coils. When a human body, or part of a human body, isplaced in the main magnetic field, the nuclear spins that are associatedwith hydrogen nuclei in tissue water become polarized. The magneticmoments that are associated with these spins become preferentiallyaligned along the direction of the main magnetic field, resulting in asmall net tissue magnetization along that axis (the “z axis,” byconvention) and the gradient coils encode the MR signal.

Acquisition, processing, analysis, and storage of medical image dataplay an important role in diagnosis and treatment of patients in ahealthcare environment. A medical imaging workflow and devices involvedin the workflow can be configured, monitored, and updated throughoutoperation of the medical imaging workflow and devices. Machine learning,deep learning, and/or other artificial intelligence can be used to helpconfigure, monitor, and update the medical imaging workflow and devices,for example.

Certain examples provide and/or facilitate improved imaging deviceswhich improve diagnostic accuracy and/or coverage. Certain examplesfacilitate improved image reconstruction and further processing toprovide improved diagnostic accuracy.

Certain examples provide improved management and analysis of medicalimages including MR images to which computer-aided diagnosis (CAD)and/or other artificial intelligence can be applied to identify andclassify anomalies/abnormalities such as prostate cancer, etc.

Certain examples improve MR imaging and image data processing technologyto enable an automated multi-part clinical analysis performingoncological scoring and CAD resulting in a patient disease (e.g.,prostate cancer, etc.) determination and routing/reporting to anotherclinical system, specialist, medical record, etc. Certain examplesprovide technological improvements to automate processing such as imagesegmentation, oncology scoring, report generation, etc., to reduce,minimize, or eliminate user interaction in the detection/diagnosisprocess.

Certain examples gather patient history and evaluate the patient'sprostate-specific antigen (PSA) level based on blood test data. PSA is asubstance produced by the prostate gland, and elevated PSA levels mayindicate prostate cancer or a non-cancerous condition such as anenlarged prostate, for example. Using image data (e.g., axial, sagittal,etc.), apparent diffusion coefficient (ADC) blood flow mappinginformation, etc., prostate gland volume and PSA density can be computedby the system, for example. Then, using, computer-aided detection and/oruser input, lesions can be identified with respect to the patient'sprostate gland using the image data, ADC information, density,segmentation, and/or other automated image data analysis, for example.Regions of interest (ROIs) can be defined around identified, possible,and/or likely lesions to mark lesion(s) in the image(s). Lesions in theROIs can then be segmented by the system (e.g., along a long axis, etc.)and scored (e.g., to determine a likelihood of lesion verification,malignancy/severity, size, etc.), for example. Deep learning, machinelearning, and/or other artificial intelligence can be used toautomatically segment and compute prostate volume and/or toautomatically segment, locate, and score lesion(s) in/on the prostategland, for example. A determination of likely prostate cancer, triggerfor patient care plan/treatment, report for urologist and/or otherclinician, etc., can be generated with score, lesion detail,observation, comment, conclusion, etc.

An apparent diffusion coefficient (ADC) image or an ADC map is an MRimage that more specifically shows diffusion than conventional diffusionweighted imaging (DWI), by eliminating certain (e.g., T2) weighing thatis otherwise inherent in conventional DWI. ADC imaging does so byacquiring multiple conventional DWI images with different amounts of DWIweighing, and the change in signal is proportional to the rate ofdiffusion.

A score, such as a pirads or pi-rads score, can represent an indicationof likely cancerous/tumor tissue, for example. PI-RADS is an acronym forProstate Imaging Reporting and Data System, defining quality standardsfor multi-parametric MR imaging including image creation and reporting.A PI-RADS score is provided for each variable parameter along a scalebased on a score of “yes” or “no for a dynamic contrast-enhanced (DCE orDice) parameter, from 1 to 5 for T2-weighted (T2 W) anddiffusion-weighted imaging (DWI), for example. The score is determinedfor each detected lesion, with 1 being most probably benign and 5 beinghighly suspicious of malignancy. For example, pirads 1 is “very low”(e.g., clinically significant cancer is highly unlikely to be present);pirads 2 is “low” (e.g., clinically significant cancer is unlikely to bepresent); pirads 3 is “intermediate” (e.g., the presence of clinicallysignificant cancer is equivocal); pirads 4 is “high” (e.g., clinicallysignificant cancer is likely to be present); and pirads 5 is “very high”(e.g., clinically significant cancer is highly likely to be present).

Machine learning techniques, whether deep learning networks or otherexperiential/observational learning system, can be used to locate anobject in an image, understand speech and convert speech into text, andimprove the relevance of search engine results, for example. Deeplearning is a subset of machine learning that uses a set of algorithmsto model high-level abstractions in data using a deep graph withmultiple processing layers including linear and non-lineartransformations. While many machine learning systems are seeded withinitial features and/or network weights to be modified through learningand updating of the machine learning network, a deep learning networktrains itself to identify “good” features for analysis. Using amultilayered architecture, machines employing deep learning techniquescan process raw data better than machines using conventional machinelearning techniques. Examining data for groups of highly correlatedvalues or distinctive themes is facilitated using different layers ofevaluation or abstraction.

Example Magnetic Resonance Imaging System

Turning to FIG. 1A, the major components of an exemplary magneticresonance imaging (MRI) system 10 are shown. Operation of the system iscontrolled from an operator console 12 which includes a keyboard orother input device 13, a control panel 14, and a display screen 16. Theconsole 12 communicates through a link 18 with a separate computersystem 20 that enables an operator to control the production and displayof images on the display screen 16. The computer system 20 includes anumber of modules which communicate with each other through a backplane20 a. These include an image processor module 22, a CPU module 24 and amemory module 26 that may include a frame buffer for storing image dataarrays. The computer system 20 is linked to archival media devices,permanent or back-up memory storage or a network for storage of imagedata and programs, and communicates with a separate system control 32through a high speed serial link 34. The input device 13 can include amouse, joystick, keyboard, track ball, touch activated screen, lightwand, voice control, or any similar or equivalent input device, and maybe used for interactive geometry prescription.

The system control 32 includes a set of modules connected together by abackplane 32 a. These include a CPU module 36 and a pulse generatormodule 38 which connects to the operator console 12 through a seriallink 40. It is through link 40 that the system control 32 receivescommands from the operator to indicate the scan sequence that is to beperformed. The pulse generator module 38 operates the system componentsto carry out the desired scan sequence and produces data which indicatesthe timing, strength and shape of the RF pulses produced, and the timingand length of the data acquisition window. The pulse generator module 38connects to a set of gradient amplifiers 42, to indicate the timing andshape of the gradient pulses that are produced during the scan. Thepulse generator module 38 can also receive patient data from aphysiological acquisition controller 44 that receives signals from anumber of different sensors connected to the patient, such as ECGsignals from electrodes attached to the patient. The pulse generatormodule 38 connects to a scan room interface circuit 46 which receivessignals from various sensors associated with the condition of thepatient and the magnet system. It is also through the scan roominterface circuit 46 that a patient positioning system 48 receivescommands to move the patient to the desired position for the scan.

Gradient waveforms produced by the pulse generator module 38 are appliedto the gradient amplifier system 42 having Gx, Gy, and Gz amplifiers.Each gradient amplifier excites a corresponding physical gradient coilin a gradient coil assembly generally designated 50 to produce themagnetic field gradients used for spatially encoding acquired signals.The gradient coil assembly 50 forms part of a magnet assembly 52 whichincludes a polarizing magnet 54 and a whole-body RF coil 56. In anembodiment of the invention, RF coil 56 is a multi-channel coil. Atransceiver module 58 in the system control 32 produces pulses which areamplified by an RF amplifier 60 and coupled to the RF coil 56 by atransmit/receive switch 62. The resulting signals emitted by the excitednuclei in the patient may be sensed by the same RF coil 56 and coupledthrough the transmit/receive switch 62 to a preamplifier 64. Theamplified MR signals are demodulated, filtered, and digitized in thereceiver section of the transceiver 58. The transmit/receive switch 62is controlled by a signal from the pulse generator module 38 toelectrically connect the RF amplifier 60 to the coil 56 during thetransmit mode and to connect the preamplifier 64 to the coil 56 duringthe receive mode. The transmit/receive switch 62 can also enable aseparate RF coil (for example, a surface coil) to be used in either thetransmit or receive mode.

MR signals received/detected by the multi-channel RF coil 56 aredigitized by the transceiver module 58 and transferred to a memorymodule 66 in the system control 32. A scan is complete when an array ofraw k-space data has been acquired in the memory module 66. This rawk-space data is rearranged into separate k-space data arrays for eachimage to be reconstructed, and each of these is input to an arrayprocessor 68 which operates to Fourier transform the data into an arrayof image data. This image data is conveyed through the serial link 34 tothe computer system 20 where it is stored in memory. In response tocommands received from the operator console 12, this image data may bearchived in long term storage or it may be further processed by theimage processor 22 and conveyed to the operator console 12 and presentedon the display 16.

Example Computer-Aided Prostate Analysis System

FIG. 1B illustrates an example computer-aided prostate analysis system100 including an image acquisition module 110, a prostate detector 120,a prostate assessor 130, a lesion identifier and assessor 140 (alsoreferred to herein as a lesion assessor), and an outcome generator 150.

The example system 100 enables computer-assisted diagnostics andclassification of prostate cancer. Certain examples analyze prostateinformation and generate a prediction and/or other analysis regardinglikely prostate cancer, malignant lesion, and/or other prostate issue.For example, certain examples position a prostate lesion on a prostatesector map using multimodal multi-protocol MR data and integrateprostate lesion information for a computer-aided diagnosis andclassification system for prostate cancer.

The example image acquisition module 110 acquires image data, such as anADC image, DWI image, and/or other MR image data, etc., for a patient.The image data includes the patient's prostate gland, for example. Theimage acquisition module 110 can preprocess the image data to prepare itfor further analysis, for example. For example, contrast, window level,etc., can be adjusted to accentuate the prostate gland in the imagedata, etc.

The example prostate detector 120 processes the image data to identifythe prostate gland in the image. For example, based on pixeldensity/intensity values, the prostate detector 120 can identify theprostate gland in the image data. In other examples, the image can besegmented and scored to identify and register the prostate gland in theimage (e.g., an MR image, 3D volume, etc.).

The example prostate assessor 130 processes the image data inconjunction with patient clinical history information and determines aprostate-specific antigen (PSA) level for the patient. An elevated PSAlevel, indicating a greater than normal presence of prostate-specificantigen in the patient's blood stream, can be an indicator of prostatecancer in the associated patient. The prostate assessor 130 can segmentthe prostate in the image and compute its volume (e.g., using deeplearning-based methods, etc.), for example. For example, the prostateassessor 130 can deposit distances (e.g., 3 distances, etc.) on theimage (e.g., using a dedicated distance tool, etc.) and prostate volumeand PSA density can be computed automatically.

The example lesion identifier and assessor MO identifies and processes alesion on the image data. For example, the lesion identifier andassessor 140 can identify and process a lesion in the image bydepositing a graphical object (e.g., indicating a region of interest(e.g., along its long axis, etc.), etc.) on a lesion in one or moreacquired images. For example, an ellipse is deposited on a prostatesector map, with schema and sector(s) underneath the map automaticallyselected (e.g., ellipses are deposited on axial, sagittal, and coronalplanes to automatically select corresponding sectors, etc.). The lesioncan then be scored by the lesion identifier and assessor MO according toPIRADS v2 guidelines. In another example, lesion(s) are automaticallysegmented and then located and scored for each available MR imagingtechnique (e.g., using non-rigid registration of the segmented prostateand a 3D model of the prostate sector map and deep learning basedmethods, etc.). A global score, for example, can be automaticallycomputed from the various MR technique lesion scores. As anotherexample, lesion(s) are identified using available tools, algorithms,digital twin, etc.

From the lesion information, a conclusion, recommendation, and/or otherevaluation regarding likely prostate issue(s) can be determined.Qualitative evaluation, hidden layer processing in a deep neuralnetwork, and an analysis of edges, edge combination(s), object models,etc., enable the deep neural network to correlate MR image data withlikely prostate lesions and/or other imperfections necessitatingfollow-up for further verification, treatment, etc. Convolution,deconvolution, forward inference and backward learning from imagesegmentation and pixel intensity data can help drive a correlationbetween MR image information and likely prostate cancer determinationvia CAD, for example.

Based on the lesion analysis, a report and/or next action trigger can begenerated and exported by the example outcome generator 150. Forexample, a report can be generated, saved, output, transferred, etc. Forexample, patient clinical history (e.g., including an identified trendin PSA level, etc.), prostate gland volume, PSA level, PSA density,lesion details, index lesion, comments, PI-RADS assessment, conclusion,etc., can be provided (e.g., transmitted to another program, triggeranother process, saved, displayed, and/or otherwise output) based on theanalysis to drive further action with respect to the patient.

Digital Twin Example

In certain examples, a digital representation of the patient, patientanatomy/region (e.g., prostate gland, etc.) can be used forcomputer-aided detection and/or diagnosis of prostate cancer. A digitalrepresentation, digital model, digital “twin”, or digital “shadow” is adigital informational construct about a physical system, process, etc.That is, digital information can be implemented as a “twin” of aphysical device/system/person/process and information associated withand/or embedded within the physical device/system/process. The digitaltwin is linked with the physical system through the lifecycle of thephysical system. In certain examples, the digital twin includes aphysical object in real space, a digital twin of that physical objectthat exists in a virtual space, and information linking the physicalobject with its digital twin. The digital twin exists in a virtual spacecorresponding to a real space and includes a link for data flow fromreal space to virtual space as well as a link for information flow fromvirtual space to real space and virtual sub-spaces.

For example, FIG. 2 illustrates a patient, prostate gland, and/or otheranatomy/anatomical region 210 in a real space 215 providing data 220 toa digital twin 230 in a virtual space 235. The digital twin 230 and/orits virtual space 235 provide information 240 back to the real space215. The digital twin 230 and/or virtual space 235 can also provideinformation to one or more virtual sub-spaces 250, 252, 254. As shown inthe example of FIG. 2, the virtual space 235 can include and/or beassociated with one or more virtual sub-spaces 250, 252, 254, which canbe used to model one or more parts of the digital twin 230 and/ordigital “sub-twins” modeling subsystems/subparts of the overall digitaltwin 230.

Sensors connected to the physical object (e.g., the patient 210) cancollect data and relay the collected data 220 to the digital twin 230(e.g., via self-reporting, using a clinical or other health informationsystem such as a picture archiving and communication system (PACS),radiology information system (RIS), electronic medical record system(EMR), laboratory information system (LIS), cardiovascular informationsystem (CVIS), hospital information system (HIS), MR imaging scanner,and/or combination thereof, etc.). Interaction between the digital twin230 and the patient/prostate 210 can help improve diagnosis, treatment,health maintenance, etc., for the patient 210 (such as identification ofprostate issues, etc.), for example. An accurate digital description 230of the patient/prostate 210 benefiting from a real-time or substantiallyreal-time (e.g., accounting from data transmission, processing, and/orstorage delay) allows the system 200 to predict “failures” in the formof disease, body function, and/or other malady, condition, etc.

In certain examples, obtained images overlaid with sensor data, labresults, etc., can be used in augmented reality (AR) applications when ahealthcare practitioner is examining, treating, and/or otherwise caringfor the patent 210. Using AR, the digital twin 230 follows the patient'sresponse to the interaction with the healthcare practitioner, forexample. Thus, the patient's prostate can be modeled to identify achange in appearance, lab results, scoring, and/or other characteristicto indicate a prostate issue such as cancer, evaluate the issue,model/predict treatment options, etc.

Thus, rather than a generic model, the digital twin 230 is a collectionof actual physics-based, anatomically-based, and/or biologically-basedmodels reflecting the patient/prostate 210 and his or her associatednorms, conditions, etc. In certain examples, three-dimensional (3D)modeling of the patient/prostate 210 creates the digital twin 230 forthe patient/prostate 210. The digital twin 230 can be used by theprostate assessor 130, for example, to determine (e.g., model, simulate,extrapolate, etc.) and view a status of the patient/prostate 210 basedon input data 220 dynamically provided from a source (e.g., from thepatient 210, imaging system, practitioner, health information system,sensor, etc.).

In certain examples, the digital twin 230 of the patient/prostate 210can be used by the prostate assessor 130 for monitoring, diagnostics,and prognostics for the patient/prostate 210. Using sensor data incombination with historical information, current and/or potential futureconditions of the patient/prostate 210 can be identified, predicted,monitored, etc., using the digital twin 230. Causation, escalation,improvement, etc., can be monitored via the digital twin 230. Using thedigital twin 230, the patient/prostate's 210 physical behaviors can besimulated and visualized for diagnosis, treatment, monitoring,maintenance, etc.

In contrast to computers, humans do not process information in asequential, step-by-step process. Instead, people try to conceptualize aproblem and understand its context. While a person can review data inreports, tables, etc., the person is most effective when visuallyreviewing a problem and trying to find its solution. Typically, however,when a person visually processes information, records the information inalphanumeric form, and then tries to re-conceptualize the informationvisually, information is lost and the problem-solving process is mademuch less efficient over time.

Using the digital twin 230, however, allows a person and/or system toview and evaluate a visualization of a situation (e.g., apatient/prostate 210 and associated patient problem, etc.) withouttranslating to data and back. With the digital twin 230 in commonperspective with the actual patient/prostate 210, physical and virtualinformation can be viewed together, dynamically and in real time (orsubstantially real time accounting for data processing, transmission,and/or storage delay). Rather than reading a report, a healthcarepractitioner can view and simulate with the digital twin 230 to evaluatea condition, progression, possible treatment, etc., for thepatient/prostate 210. In certain examples, features, conditions, trends,indicators, traits, etc., can be tagged and/or otherwise labeled in thedigital twin 230 to allow the practitioner to quickly and easily viewdesignated parameters, values, trends, alerts, etc.

The digital twin 230 can also be used for comparison (e.g., to thepatient/prostate 210, to a “normal”, standard, or reference patient, setof clinical criteria/symptoms, best practices, protocol steps, etc.). Incertain examples, the digital twin 230 of the patient/prostate 210 canbe used to measure and visualize an ideal or “gold standard” value statefor that patient/protocol/item, a margin for error or standard deviationaround that value (e.g., positive and/or negative deviation from thegold standard value, etc.), an actual value, a trend of actual values,etc. A difference between the actual value or trend of actual values andthe gold standard (e.g., that falls outside the acceptable deviation)can be visualized as an alphanumeric value, a color indication, apattern, etc.

Further, the digital twin 230 of the patient 210 can facilitatecollaboration among friends, family, care providers, etc., for thepatient 210. Using the digital twin 230, conceptualization of thepatient 210 and his/her health can be shared (e.g., according to a careplan, etc.) among multiple people including care providers, family,friends, etc. People do not need to be in the same location as thepatient 210, with each other, etc., and can still view, interact with,and draw conclusions from the same digital twin 230, for example.

Thus, the digital twin 230 can be defined as a set of virtualinformation constructs that describes (e.g., fully describes) thepatient 210 from a micro level (e.g., heart, lungs, foot, prostategland, anterior cruciate ligament (ACL), stroke history, etc.) to amacro level (e.g., whole anatomy, holistic view, skeletal system,nervous system, vascular system, etc.). Similarly, the digital twin 230can represent an item and/or a protocol at various levels of detail suchas macro, micro, etc. In certain examples, the digital twin 230 can be areference digital twin (e.g., a digital twin prototype, etc.) and/or adigital twin instance. The reference digital twin represents aprototypical or “gold standard” model of the patient/prostate 210 or ofa particular type/category of patient/prostate 210, while one or morereference digital twins represent particular patient(s)/prostate(s) 210.Thus, the digital twin 230 of a child patient 210 may be implemented asa child reference digital twin organized according to certain standardor “typical” child characteristics, with a particular digital twininstance representing the particular child patient 210. In certainexamples, multiple digital twin instances can be aggregated into adigital twin aggregate (e.g., to represent an accumulation orcombination of multiple child patients sharing a common referencedigital twin, etc.). The digital twin aggregate can be used to identifydifferences, similarities, trends, etc., between children represented bythe child digital twin instances, for example.

In certain examples, the virtual space 235 in which the digital twin 230(and/or multiple digital twin instances, etc.) operates is referred toas a digital twin environment. The digital twin environment 235 providesan integrated, multi-domain physics- and/or biologics-based applicationspace in which to operate the digital twin 230. The digital twin 230 canbe analyzed in the digital twin environment 235 to predict futurebehavior, condition, progression, etc., of the patient/protocol/item210, for example. The digital twin 230 can also be interrogated orqueried in the digital twin environment 235 to retrieve and/or analyzecurrent information 240, past history, etc.

In certain examples, the digital twin environment 235 can be dividedinto multiple virtual spaces 250-254. Each virtual space 250-254 canmodel a different digital twin instance and/or component of the digitaltwin 230 and/or each virtual space 250-254 can be used to perform adifferent analysis, simulation, etc., of the same digital twin 230.Using the multiple virtual spaces 250-254, the digital twin 230 can betested inexpensively and efficiently in a plurality of ways whilepreserving patient 210 safety. A healthcare provider can then understandhow the patient/prostate 210 may react to a variety of treatments in avariety of scenarios, for example. Continuous, triggered, periodic,and/or other input 260 from the real space to the virtual space enablesthe digital twin 230 to continue to evolve.

Example Deep Learning and Other Machine Learning

Deep learning is a class of machine learning techniques employingrepresentation learning methods that allows a machine to be given rawdata and determine the representations needed for data classification.Deep learning ascertains structure in data sets using back propagationalgorithms which are used to alter internal parameters (e.g., nodeweights) of the deep learning machine. Deep learning machines canutilize a variety of multilayer architectures and algorithms. Whilemachine learning, for example, involves an identification of features tobe used in training the network, deep learning processes raw data toidentify features of interest without the external identification.

Deep learning in a neural network environment includes numerousinterconnected nodes referred to as neurons. Input neurons, activatedfrom an outside source, activate other neurons based on connections tothose other neurons which are governed by the machine parameters. Aneural network behaves in a certain manner based on its own parameters.Learning refines the machine parameters, and, by extension, theconnections between neurons in the network, such that the neural networkbehaves in a desired manner.

Deep learning that utilizes a convolutional neural network segments datausing convolutional filters to locate and identify learned, observablefeatures in the data. Each filter or layer of the CNN architecturetransforms the input data to increase the selectivity and invariance ofthe data. This abstraction of the data allows the machine to focus onthe features in the data it is attempting to classify and ignoreirrelevant background information.

Deep learning operates on the understanding that many datasets includehigh level features which include low level features. While examining animage, for example, rather than looking for an object, it is moreefficient to look for edges which form motifs which form parts, whichform the object being sought. These hierarchies of features can be foundin many different forms of data such as speech and text, etc.

Learned observable features include objects and quantifiableregularities learned by the machine during supervised learning. Amachine provided with a large set of well classified data is betterequipped to distinguish and extract the features pertinent to successfulclassification of new data.

A deep learning machine that utilizes transfer learning may properlyconnect data features to certain classifications affirmed by a humanexpert. Conversely, the same machine can, when informed of an incorrectclassification by a human expert, update the parameters forclassification. Settings and/or other configuration information, forexample, can be guided by learned use of settings and/or otherconfiguration information, and, as a system is used more (e.g.,repeatedly and/or by multiple users), a number of variations and/orother possibilities for settings and/or other configuration informationcan be reduced for a given situation.

An example deep learning neural network can be trained on a set ofexpert classified data, for example. This set of data builds the firstparameters for the neural network, and this would be the stage ofsupervised learning. During the stage of supervised learning, the neuralnetwork can be tested whether the desired behavior has been achieved.

Once a desired neural network behavior has been achieved (e.g., amachine has been trained to operate according to a specified threshold,etc.), the machine can be deployed for use (e.g., testing the machinewith “real” data, etc.). During operation, neural networkclassifications can be confirmed or denied (e.g., by an expert user,expert system, reference database, etc.) to continue to improve neuralnetwork behavior. The example neural network is then in a state oftransfer learning, as parameters for classification that determineneural network behavior are updated based on ongoing interactions. Incertain examples, the neural network can provide direct feedback toanother process. In certain examples, the neural network outputs datathat is buffered (e.g., via the cloud, etc.) and validated before it isprovided to another process.

Deep learning machines using convolutional neural networks (CNNs) can beused for image analysis. Stages of CNN analysis can be used for facialrecognition in natural images, identification of lesions in image data,computer-aided diagnosis (CAD), etc.

High quality medical image data can be acquired using one or moreimaging modalities, such as x-ray, computed tomography (CT), molecularimaging and computed tomography (MET), magnetic resonance imaging (MRI),etc. Medical image quality is often not affected by the machinesproducing the image but the patient. A patient moving during an MRI cancreate a blurry or distorted image that can prevent accurate diagnosis,for example.

Interpretation of medical images, regardless of quality, is only arecent development. Medical images are largely interpreted byphysicians, but these interpretations can be subjective, affected by thecondition of the physician's experience in the field and/or fatigue.Image analysis via machine learning can support a healthcarepractitioner's workflow.

Deep learning machines can provide computer aided detection support toimprove their image analysis with respect to image quality andclassification, for example. However, issues facing deep learningmachines applied to the medical field often lead to numerous falseclassifications. Deep learning machines must overcome small trainingdatasets and require repetitive adjustments, for example.

Deep learning machines, with minimal training, can be used to determinethe quality of a medical image, for example. Semi-supervised andunsupervised deep learning machines can be used to quantitativelymeasure qualitative aspects of images. For example, deep learningmachines can be utilized after an image has been acquired to determineif the quality of the image is sufficient for diagnosis. Supervised deeplearning machines can also be used for computer aided diagnosis. Forexample, the lesion identifier and assessor 140 can use a deep learningnetwork model to analyze lesion data identified in an image. Theprostate assessor 130 can utilize a deep learning network model toevaluate prostate health based on an identified prostate gland in animage and associated patient health information, for example. Supervisedlearning can help reduce susceptibility to false classification, forexample.

Deep learning machines can utilize transfer learning when interactingwith physicians to counteract the small dataset available in thesupervised training. These deep learning machines can improve theircomputer aided diagnosis over time through training and transferlearning. In certain examples, the digital twin 230 (e.g., as a wholeand/or in one of its sub-parts 250-254) can leverage a deep learningnetwork model to model behavior of a component, such as a prostategland, lesion, other organ, etc.

Example Learning Network Systems

FIG. 3 is a representation of an example learning neural network 300.The example neural network 300 includes layers 320, 340, 360, and 380.The layers 320 and 340 are connected with neural connections 330. Thelayers 340 and 360 are connected with neural connections 350. The layers360 and 380 are connected with neural connections 370. Data flowsforward via inputs 312, 314, 316 from the input layer 320 to the outputlayer 380 and to an output 390.

The layer 320 is an input layer that, in the example of FIG. 3, includesa plurality of nodes 322, 324, 326. The layers 340 and 360 are hiddenlayers and include, the example of FIG. 3, nodes 342, 344, 346, 348,362, 364, 366, 368. The neural network 300 may include more or lesshidden layers 340 and 360 than shown. The layer 380 is an output layerand includes, in the example of FIG. 3, a node 382 with an output 390.Each input 312-316 corresponds to a node 322-326 of the input layer 320,and each node 322-326 of the input layer 320 has a connection 330 toeach node 342-348 of the hidden layer 340. Each node 342-348 of thehidden layer 340 has a connection 350 to each node 362-368 of the hiddenlayer 360. Each node 362-368 of the hidden layer 360 has a connection370 to the output layer 380. The output layer 380 has an output 390 toprovide an output from the example neural network 300.

Of connections 330, 350, and 370 certain example connections 332, 352,372 may be given added weight while other example connections 334, 354,374 may be given less weight in the neural network 300. Input nodes322-326 are activated through receipt of input data via inputs 312-316,for example. Nodes 342-348 and 362-368 of hidden layers 340 and 360 areactivated through the forward flow of data through the network 300 viathe connections 330 and 350, respectively. Node 382 of the output layer380 is activated after data processed in hidden layers 340 and 360 issent via connections 370. When the output node 382 of the output layer380 is activated, the node 382 outputs an appropriate value based onprocessing accomplished in hidden layers 340 and 360 of the neuralnetwork 300.

FIG. 4 illustrates a particular implementation of the example neuralnetwork 300 as a convolutional neural network 400. As shown in theexample of FIG. 4, an input 310 is provided to the first layer 320 whichprocesses and propagates the input 310 to the second layer 340. Theinput 310 is further processed in the second layer 340 and propagated tothe third layer 360. The third layer 360 categorizes data to be providedto the output layer e80. More specifically, as shown in the example ofFIG. 4, a convolution 404 (e.g., a 5×5 convolution, etc.) is applied toa portion or window (also referred to as a “receptive field”) 402 of theinput 310 (e.g., a 32×32 data input, etc.) in the first layer 320 toprovide a feature map 406 (e.g., a (6×) 28×28 feature map, etc.). Theconvolution 404 maps the elements from the input 310 to the feature map406. The first layer 320 also provides subsampling (e.g., 2×2subsampling, etc.) to generate a reduced feature map 410 (e.g., a (6×)14×14 feature map, etc). The feature map 410 undergoes a convolution 412and is propagated from the first layer 320 to the second layer 340,where the feature map 410 becomes an expanded feature map 414 (e.g., a(16×) 10×10 feature map, etc.). After subsampling 416 in the secondlayer 340, the feature map 414 becomes a reduced feature map 418 (e.g.,a (16×) 4×5 feature map, etc.). The feature map 418 undergoes aconvolution 420 and is propagated to the third layer 360, where thefeature map 418 becomes a classification layer 422 forming an outputlayer of N categories 424 with connection 426 to the convoluted layer422, for example.

FIG. 5 is a representation of an example implementation of an imageanalysis convolutional neural network 500. The convolutional neuralnetwork 500 receives an input image 502 and abstracts the image in aconvolution layer 504 to identify learned features 510-522. In a secondconvolution layer 530, the image is transformed into a plurality ofimages 530-538 in which the learned features 510-522 are eachaccentuated in a respective sub-image 530-538. The images 530-538 arefurther processed to focus on the features of interest 510-522 in images540-548. The resulting images 540-548 are then processed through apooling layer which reduces the size of the images 540-548 to isolateportions 550-554 of the images 540-548 including the features ofinterest 510-522. Outputs 550-554 of the convolutional neural network500 receive values from the last non-output layer and classify the imagebased on the data received from the last non-output layer. In certainexamples, the convolutional neural network 500 may contain manydifferent variations of convolution layers, pooling layers, learnedfeatures, and outputs, etc.

FIG. 6A illustrates an example configuration 600 to apply a learning(e.g., machine learning, deep learning, etc.) network to process and/orotherwise evaluate an image. Machine learning can be applied to avariety of processes including image acquisition, image reconstruction,image analysis/diagnosis, etc. As shown in the example configuration 600of FIG. 6A, raw data 610 (e.g., raw data 610 such as sonogram raw data,etc., obtained from an imaging scanner such as an x-ray, computedtomography, ultrasound, magnetic resonance, etc., scanner) is fed into alearning network 620. The learning network 620 processes the data 610 tocorrelate and/or otherwise combine the raw image data 620 into aresulting image 630 (e.g., a “good quality” image and/or other imageproviding sufficient quality for diagnosis, etc.). The learning network620 includes nodes and connections (e.g., pathways) to associate rawdata 610 with a finished image 630. The learning network 620 can be atraining network that learns the connections and processes feedback toestablish connections and identify patterns, for example. The learningnetwork 620 can be a deployed network that is generated from a trainingnetwork and leverages the connections and patterns established in thetraining network to take the input raw data 610 and generate theresulting image 630, for example.

Once the learning 620 is trained and produces good images 630 from theraw image data 610, the network 620 can continue the “self-learning”process and refine its performance as it operates. For example, there is“redundancy” in the input data (raw data) 610 and redundancy in thenetwork 620, and the redundancy can be exploited.

If weights assigned to nodes in the learning network 620 are examined,there are likely many connections and nodes with very low weights. Thelow weights indicate that these connections and nodes contribute littleto the overall performance of the learning network 620. Thus, theseconnections and nodes are redundant. Such redundancy can be evaluated toreduce redundancy in the inputs (raw data) 610. Reducing input 610redundancy can result in savings in scanner hardware, reduced demands oncomponents, and also reduced exposure dose to the patient, for example.

In deployment, the configuration 600 forms a package 600 including aninput definition 610, a trained network 620, and an output definition630. The package 600 can be deployed and installed with respect toanother system, such as an imaging system, analysis engine, etc.

As shown in the example of FIG. 6B, the learning network 620 can bechained and/or otherwise combined with a plurality of learning networks621-623 to form a larger learning network. The combination of networks620-623 can be used to further refine responses to inputs and/orallocate networks 620-623 to various aspects of a system, for example.

In some examples, in operation, “weak” connections and nodes caninitially be set to zero. The learning network 620 then processes itsnodes in a retaining process. In certain examples, the nodes andconnections that were set to zero are not allowed to change during theretraining. Given the redundancy present in the network 620, it ishighly likely that equally good images will be generated. As illustratedin FIG. 6B, after retraining, the learning network 620 becomes DLN 621.The learning network 621 is also examined to identify weak connectionsand nodes and set them to zero. This further retrained network islearning network 622. The example learning network 622 includes the“zeros” in learning network 621 and the new set of nodes andconnections. The learning network 622 continues to repeat the processinguntil a good image quality is reached at a learning network 623, whichis referred to as a “minimum viable net (MVN)”. The learning network 623is a MVN because if additional connections or nodes are attempted to beset to zero in learning network 623, image quality can suffer.

Once the MVN has been obtained with the learning network 623, “zero”regions (e.g., dark irregular regions in a graph) are mapped to theinput 610. Each dark zone is likely to map to one or a set of parametersin the input space. For example, one of the zero regions may be linkedto the number of views and number of channels in the raw data. Sinceredundancy in the network 623 corresponding to these parameters can bereduced, there is a highly likelihood that the input data can be reducedand generate equally good output. To reduce input data, new sets of rawdata that correspond to the reduced parameters are obtained and runthrough the learning network 621. The network 620-623 may or may not besimplified, but one or more of the learning networks 620-623 isprocessed until a “minimum viable input (MVI)” of raw data input 610 isreached. At the MVI, a further reduction in the input raw data 610 mayresult in reduced image 630 quality. The MVI can result in reducedcomplexity in data acquisition, less demand on system components,reduced stress on patients (e.g., less breath-hold or contrast), and/orreduced dose to patients, for example.

By forcing some of the connections and nodes in the learning networks620-623 to zero, the network 620-623 to build “collaterals” tocompensate. In the process, insight into the topology of the learningnetwork 620-623 is obtained. Note that network 621 and network 622, forexample, have different topology since some nodes and/or connectionshave been forced to zero. This process of effectively removingconnections and nodes from the network extends beyond “deep learning”and can be referred to as “deep-deep learning”, for example.

In certain examples, input data processing and deep learning stages canbe implemented as separate systems. However, as separate systems,neither module may be aware of a larger input feature evaluation loop toselect input parameters of interest/importance. Since input dataprocessing selection matters to produce high-quality outputs, feedbackfrom deep learning systems can be used to perform input parameterselection optimization or improvement via a model. Rather than scanningover an entire set of input parameters to create raw data (e.g., whichis brute force and can be expensive), a variation of active learning canbe implemented. Using this variation of active learning, a startingparameter space can be determined to produce desired or “best” resultsin a model. Parameter values can then be randomly decreased to generateraw inputs that decrease the quality of results while still maintainingan acceptable range or threshold of quality and reducing runtime byprocessing inputs that have little effect on the model's quality.

FIG. 7 illustrates example training and deployment phases of a learningnetwork, such as a deep learning or other machine learning network. Asshown in the example of FIG. 7, in the training phase, a set of inputs702 is provided to a network 704 for processing. In this example, theset of inputs 702 can include facial features of an image to beidentified. The network 704 processes the input 702 in a forwarddirection 706 to associate data elements and identify patterns. Thenetwork 704 determines that the input 702 represents a dog 708. Intraining, the network result 708 is compared 710 to a known outcome 712.In this example, the known outcome 712 is a human face (e.g., the inputdata set 702 represents a human face, not a dog face). Since thedetermination 708 of the network 704 does not match 710 the knownoutcome 712, an error 714 is generated. The error 714 triggers ananalysis of the known outcome 712 and associated data 702 in reversealong a backward pass 716 through the network 704. Thus, the trainingnetwork 704 learns from forward 706 and backward 716 passes with data702, 712 through the network 704.

Once the comparison of network output 708 to known output 712 matches710 according to a certain criterion or threshold (e.g., matches ntimes, matches greater than x percent, etc.), the training network 704can be used to generate a network for deployment with an externalsystem. Once deployed, a single input 720 is provided to a deployedlearning network 722 to generate an output 724. In this case, based onthe training network 704, the deployed network 722 determines that theinput 720 is an image of a human face 724.

FIG. 8 illustrates an example product leveraging a trained networkpackage to provide a deep and/or other machine learning productoffering. As shown in the example of FIG. 8, an input 810 (e.g., rawdata) is provided for preprocessing 820. For example, the raw input data810 is preprocessed 820 to check format, completeness, etc. Once thedata 810 has been preprocessed 820, patches are created 830 of the data.For example, patches or portions or “chunks” of data are created 830with a certain size and format for processing. The patches are then fedinto a trained network 840 for processing. Based on learned patterns,nodes, and connections, the trained network 840 determines outputs basedon the input patches. The outputs are assembled 850 (e.g., combinedand/or otherwise grouped together to generate a usable output, etc.).The output is then displayed 860 and/or otherwise output to a user(e.g., a human user, a clinical system, an imaging modality, a datastorage (e.g., cloud storage, local storage, edge device, etc.), etc.).

As discussed above, learning networks can be packaged as devices fortraining, deployment, and application to a variety of systems. FIGS.9A-9C illustrate various learning device configurations. For example,FIG. 9A shows a general learning device 900. The example device 900includes an input definition 910, a learning network model 920, andoutput definitions 930. The input definition 910 can include one or moreinputs translating into one or more outputs 930 via the network 920.

FIG. 9B shows an example training device 901. That is, the trainingdevice 901 is an example of the device 900 configured as a traininglearning network device. In the example of FIG. 9B, a plurality oftraining inputs 911 are provided to a network 921 to develop connectionsin the network 921 and provide an output to be evaluated by an outputevaluator 931. Feedback is then provided by the output evaluator 931into the network 921 to further develop (e.g., train) the network 921.Additional input 911 can be provided to the network 921 until the outputevaluator 931 determines that the network 921 is trained (e.g., theoutput has satisfied a known correlation of input to output according toa certain threshold, margin of error, etc.).

FIG. 9C depicts an example deployed device 903. Once the training device901 has learned to a requisite level, the training device 901 can bedeployed for use. While the training device 901 processes multipleinputs to learn, the deployed device 903 processes a single input todetermine an output, for example. As shown in the example of FIG. 9C,the deployed device 903 includes an input definition 913, a trainednetwork 923, and an output definition 933. The trained network 923 canbe generated from the network 921 once the network 921 has beensufficiently trained, for example. The deployed device 903 receives asystem input 913 and processes the input 913 via the network 923 togenerate an output 933, which can then be used by a system with whichthe deployed device 903 has been associated, for example.

Example Image Analysis and Prostate Evaluation Systems and Methods

Certain examples provide systems and methods for computer-assisteddiagnostics and classification of prostate cancer. For example, certainexamples position a prostate lesion on a prostate sector map usingmultimodal multi-protocol MR data and integrate prostate lesioninformation for a computer-aided diagnosis and classification system forprostate cancer.

For example, in a first workflow, a graphical object (e.g., ROI/longaxis) is deposited on a lesion in one or more acquired images.Additionally, an ellipse is deposited on a prostate sector map, withschema and sector(s) underneath the map automatically selected. Thelesion can then be scored according to PIRADS v2 guidelines. Based onthe lesion mapping and score, a report and/or next action trigger can begenerated and exported.

In another workflow, for example, MR image acquisition is performed, andresulting image(s) are loaded and displayed. Patient clinical history isobtained (e.g., from a clinician, patient, electronic medical record,etc.), and the patient's PSA level is determined. The prostate isautomatically segmented and its volume is computed (e.g., using deeplearning-based methods, etc.). A graphical object (e.g., ROI/long axis)is deposited on the MR data, and corresponding sector(s) is(are)automatically selected (e.g., using non-rigid registration of thesegmented prostate and a three-dimensional (3D) model of the prostatesector map, etc.). Lesion(s) can then be scored according to PIRADS v2guidelines. Based on the region analysis and lesion score, a reportand/or next action trigger can be generated and exported.

In another workflow, for example, MR image acquisition is performed, andresulting image(s) are loaded and displayed. Patient clinical history isobtained (e.g., from a clinician, patient, electronic medical record,etc.), and the patient's PSA level is determined. The prostate isautomatically segmented and its volume is computed (e.g., using deeplearning-based methods, etc.). Lesion(s) are automatically segmented andthen located and scored for each available MR imaging technique (e.g.,using non-rigid registration of the segmented prostate and a 3D model ofthe prostate sector map and deep learning based methods, etc.). Based onthe lesion segmentation, analysis and score, a report and/or next actiontrigger can be generated and exported.

In certain examples, a deep learning network model can process the imagedata to generate a binary mask output to identify a lesion on theprostate gland in the image(s). The model can take one or more imageslices, a three-dimensional volume, etc. (e.g., that has beenpre-processed to normalize intensity and/or resolution, etc.), andsegment the image data via the network to provide a binary maskidentifying the lesion in the image data. The lesion can be positionedon a prostate sector map using multimodal multi-protocol MR data via thenetwork model, for example.

Thus, certain examples provide processing, review, analysis, andcommunication of 3D reconstructed images and their relationship tooriginally acquired images from MR scanning devices. A combination ofacquired images, reconstructed images, annotations, and measurementsperformed by the clinician and/or automatically using deep learningand/or other artificial intelligence provide a referring physician withclinically relevant information that can aid in diagnosis and treatmentplanning, for example.

FIG. 10 illustrates an example method and associated infrastructure toanalyze prostate information and generate a prediction and/or otheranalysis regarding likely prostate cancer, malignant lesion, and/orother prostate issue. At block 1002, patient clinical history and PSAlevel are determined (see, e.g., the example interface of FIG. 11). Atblock 1004, prostate gland volume is computed. For example, distances(e.g., 3 distances, etc.) are deposited on the image (e.g., using adedicated distance tool, etc.) and prostate volume and PSA density arecomputed automatically (see, e.g., the example interface of FIG. 12).For example, the prostate volume can be automatically computed usingthree distances drawn on the prostate in the image via the userinterface to mark the length (d1), width (d2), and height (d3) of theprostate gland in the image. The prostate gland volume can then becomputed as length×width×height×0.52=prostate gland volume, where 0.52is an example of a scaling factor to account for differences betweenactual size and representation in the image data. PSA density can thenbe computed from the prostate gland volume and other factors, forexample.

At block 1006, lesion(s) are identified and assessed. Lesion(s) can beidentified and analyzed in a plurality of implementations. For example,a new lesion can be added (e.g., labeled) on MR image(s) (see, e.g., at1302 in the example interface of FIG. 13). For example, a long axisdistance and an ADC region of interest (ROI) can be deposited on theimage(s) via the interface. As another example, lesion(s) are identifiedusing available tools, algorithms, digital twin, etc. (see, e.g., 1402in the example interface of FIG. 14). Another example of lesion locationdetermination 1502 is shown in the example graphical user interface ofFIG. 15. As shown in the example interface of FIG. 16, ellipses aredeposited on axial, sagittal, and coronal planes in the interface 1602to automatically select corresponding sectors. For example, once anellipse is positioned on a prostate sector map schema, sector(s)underneath the ellipse are automatically selected. In FIG. 17, lesionsare scored on each available MR technique, and a global score isautomatically computed from the MR technique lesion scores, for example.For example, a lesion score can be based on its size (e.g., length,width, volume, etc.), position, etc., and scores can include aT1-weighted pulse sequence score, a T2-weighted pulse sequence score, adiffusion-weighted imaging (DWI) score, a dynamic contrast-enhanced(DCE) MRI score, an overall score, etc.

At block 1008, a report can be generated, saved, output, transferred,etc. (see, e.g., the example interface of FIG. 18). FIGS. 19A-19Cillustrate an example report showing prostate evaluation, score, PI-RADSassessment, ADC information, etc. For example, patient clinical history(e.g., including an identified trend in PSA level, etc.), prostate glandvolume, PSA level, PSA density, lesion details, index lesion, comments,PI-RADS assessment, conclusion, etc., can be provided (e.g., transmittedto another program, trigger another process, saved, displayed, and/orotherwise output) based on the analysis to drive further action withrespect to the patient.

Thus, axial and sagittal MR image views can be used in a training set aswell as an evaluation set to develop and test a deep learning network,such as the network 300, 400, 500, to analyze MR prostate image data andidentify and classify lesion(s) in the image. From the lesioninformation, a conclusion, recommendation, and/or other evaluationregarding likely prostate issue(s) can be determined. Qualitativeevaluation, hidden layer processing in a deep neural network, and ananalysis of edges, edge combination(s), object models, etc enable thedeep neural network to correlate MR image data with likely prostatelesions and/or other imperfections necessitating follow-up for furtherverification, treatment, etc. Convolution, deconvolution, forwardinference and backward learning from image segmentation and pixelintensity data can help drive a correlation between MR image informationand likely prostate cancer determination via CAD, for example.

While example implementations are illustrated in conjunction with FIGS.1-19C, elements, processes and/or devices illustrated in conjunctionwith FIGS. 1-19C may be combined, divided, re-arranged, omitted,eliminated and/or implemented in any other way. Further, componentsdisclosed and described herein can be implemented by hardware, machinereadable instructions, software, firmware and/or any combination ofhardware, machine readable instructions, software and/or firmware. Thus,for example, components disclosed and described herein can beimplemented by analog and/or digital circuit(s), logic circuit(s),programmable processor(s), application specific integrated circuit(s)(ASIC(s)), programmable logic device(s) (PLD(s)) and/or fieldprogrammable logic device(s) (FPLD(s)). When reading any of theapparatus or system claims of this patent to cover a purely softwareand/or firmware implementation, at least one of the components is/arehereby expressly defined to include a tangible computer readable storagedevice or storage disk such as a memory, a digital versatile disk (DVD),a compact disk (CD), a Blu-ray disk, etc. storing the software and/orfirmware.

Flowcharts representative of example machine readable instructions forimplementing components disclosed and described herein are shown inconjunction with at least FIG. 10. In the examples, the machine readableinstructions include a program for execution by a processor such as theprocessor 2012 shown in the example processor platform 2000 discussedbelow in connection with FIG. 20. The program may be embodied in machinereadable instructions stored on a tangible computer readable storagemedium such as a CD-ROM, a floppy disk, a hard drive, a digitalversatile disk (DVD), a Blu-ray disk, or a memory associated with theprocessor 2012, but the entire program and/or parts thereof couldalternatively be executed by a device other than the processor 2012and/or embodied in firmware or dedicated hardware. Further, although theexample program is described with reference to the flowchartsillustrated in conjunction with at least FIG. 10, many other methods ofimplementing the components disclosed and described herein mayalternatively be used. For example, the order of execution of the blocksmay be changed, and/or some of the blocks described may be changed,eliminated, or combined. Although the flowcharts of at least FIG. 10depict example operations in an illustrated order, these operations arenot exhaustive and are not limited to the illustrated order. Inaddition, various changes and modifications may be made by one skilledin the art within the spirit and scope of the disclosure. For example,blocks illustrated in the flowchart may be performed in an alternativeorder or may be performed in parallel.

As mentioned above, the example processes of at least FIG. 10 may beimplemented using coded instructions (e.g., computer and/or machinereadable instructions) stored on a tangible computer readable storagemedium such as a hard disk drive, a flash memory, a read-only memory(ROM), a compact disk (CD), a digital versatile disk (DVD), a cache, arandom-access memory (RAM) and/or any other storage device or storagedisk in which information is stored for any duration (e.g., for extendedtime periods, permanently, for brief instances, for temporarilybuffering, and/or for caching of the information). As used herein, theterm tangible computer readable storage medium is expressly defined toinclude any type of computer readable storage device and/or storage diskand to exclude propagating signals and to exclude transmission media. Asused herein, “tangible computer readable storage medium” and “tangiblemachine readable storage medium” are used interchangeably. Additionallyor alternatively, the example processes of at least FIG. 10 may beimplemented using coded instructions (e.g., computer and/or machinereadable instructions) stored on a non-transitory computer and/ormachine readable medium such as a hard disk drive, a flash memory, aread-only memory, a compact disk, a digital versatile disk, a cache, arandom-access memory and/or any other storage device or storage disk inwhich information is stored for any duration (e.g., for extended timeperiods, permanently, for brief instances, for temporarily buffering,and/or for caching of the information). As used herein, the termnon-transitory computer readable medium is expressly defined to includeany type of computer readable storage device and/or storage disk and toexclude propagating signals and to exclude transmission media. As usedherein, when the phrase “at least” is used as the transition term in apreamble of a claim, it is open-ended in the same manner as the term“comprising” is open ended. In addition, the term “including” isopen-ended in the same manner as the term “comprising” is open-ended.

FIG. 20 is a block diagram of an example processor platform 2000structured to executing the instructions of at least FIG. 10 toimplement the example components disclosed and described herein. Theprocessor platform 2000 can be, for example, a server, a personalcomputer, a mobile device (e.g., a cell phone, a smart phone, a tabletsuch as an iPad™), a personal digital assistant (PDA), an Internetappliance, or any other type of computing device.

The processor platform 2000 of the illustrated example includes aprocessor 2012. The processor 2012 of the illustrated example ishardware. For example, the processor 2012 can be implemented byintegrated circuits, logic circuits, microprocessors or controllers fromany desired family or manufacturer.

The processor 2012 of the illustrated example includes a local memory2013 (e.g., a cache). The example processor 2012 of FIG. 20 executes theinstructions of at least FIG. 10 to implement the systems andinfrastructure and associated methods of FIGS. 1-19C such as an imageacquisition module, a prostate detector, a prostate assessor, a lesionidentifier and assessor, an outcome generator, etc. The processor 2012of the illustrated example is in communication with a main memoryincluding a volatile memory 2014 and a non-volatile memory 2016 via abus 2018. The volatile memory 2014 may be implemented by SynchronousDynamic Random Access Memory (SDRAM), Dynamic Random Access Memory(DRAM), RAMBUS Dynamic Random Access Memory (RDRAM) and/or any othertype of random access memory device. The non-volatile memory 2016 may beimplemented by flash memory and/or any other desired type of memorydevice. Access to the main memory 2014, 2016 is controlled by a clockcontroller.

The processor platform 2000 of the illustrated example also includes aninterface circuit 2020. The interface circuit 2020 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), and/or a PCI express interface.

In the illustrated example, one or more input devices 2022 are connectedto the interface circuit 2020. The input device(s) 2022 permit(s) a userto enter data and commands into the processor 2012. The input device(s)can be implemented by, for example, a sensor, a microphone, a camera(still or video), a keyboard, a button, a mouse, a touchscreen, atrack-pad, a trackball, isopoint and/or a voice recognition system.

One or more output devices 2024 are also connected to the interfacecircuit 2020 of the illustrated example. The output devices 2024 can beimplemented, for example, by display devices (e.g., a light emittingdiode (LED), an organic light emitting diode (OLED), a liquid crystaldisplay, a cathode ray tube display (CRT), a touchscreen, a tactileoutput device, and/or speakers). The interface circuit 2020 of theillustrated example, thus, typically includes a graphics driver card, agraphics driver chip or a graphics driver processor.

The interface circuit 2020 of the illustrated example also includes acommunication device such as a transmitter, a receiver, a transceiver, amodem and/or network interface card to facilitate exchange of data withexternal machines (e.g., computing devices of any kind) via a network2026 (e.g., an Ethernet connection, a digital subscriber line (DSL), atelephone line, coaxial cable, a cellular telephone system, etc.).

The processor platform 2000 of the illustrated example also includes oneor more mass storage devices 2028 for storing software and/or data.Examples of such mass storage devices 2028 include floppy disk drives,hard drive disks, compact disk drives, Blu-ray disk drives, RAIDsystems, and digital versatile disk (DVD) drives.

The coded instructions 2032 of FIG. 20 may be stored in the mass storagedevice 2028, in the volatile memory 2014, in the non-volatile memory2016, and/or on a removable tangible computer readable storage mediumsuch as a CD or DVD.

From the foregoing, it will be appreciated that the above disclosedmethods, apparatus, and articles of manufacture have been disclosed tomonitor, process, and improve operation of imaging and/or otherhealthcare systems, associated/included processors/computing devices,and resulting computer-aided prostate diagnosis using a plurality ofdeep learning and/or other machine learning techniques in conjunctionwith imaging data for a patient. Certain examples provide an automatedand/or guided workflow and associated systems leveraging artificialintelligence networks and/or other systems to determine patient history,prostate gland volume, lesion identification and assessment, andrecommendation/reporting. Certain examples associate a lesion with asector map of a prostate and automate segmentation of the prostate glandand lesion. Artificial intelligence enables PIRADS and/or other scoringto develop a computer-assisted diagnosis and/or next action(s) infurther diagnosis, treatment, reporting, triggering, etc. While MRreading time can be lengthy and difficult, certain examples automate MRimage analysis and/or assist a user in evaluating relevant informationemphasized in the image(s). Additionally, automated analyses can help toreduce an amount of unnecessary prostate biopsies while improving earlydetection, treatment, and monitoring of prostate issues.

Although certain example methods, apparatus and articles of manufacturehave been described herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all methods,apparatus and articles of manufacture fairly falling within the scope ofthe claims of this patent.

What is claimed is:
 1. A computer-aided prostate condition diagnosisapparatus comprising: a memory to store instructions; and a processor toexecute the instructions to implement at least: a prostate assessor toevaluate a volume and density of a prostate gland in an image of apatient to determine a prostate-specific antigen level for the prostategland; a lesion assessor to analyze a lesion on the prostate gland inthe image; and an outcome generator to generate an assessment ofprostate gland health based on the prostate-specific antigen level andthe analysis of the lesion.
 2. The apparatus of claim 1, wherein thelesion assessor is to analyze the lesion using a plurality of scores toform a global score to classify the lesion.
 3. The apparatus of claim 1,wherein the prostate assessor is to evaluate the volume and density ofthe prostate gland by segmenting and modeling the image using a deeplearning network model.
 4. The apparatus of claim 1, wherein the lesionassessor is to analyze the lesion by segmenting and modeling the imageusing a deep learning network model.
 5. The apparatus of claim 1,wherein the lesion assessor is to generate an ellipse to be overlaid onthe lesion, the overlay of the ellipse to trigger automatic selection ofone or more sectors underneath the ellipse in the image.
 6. Theapparatus of claim 1, wherein the image includes a three-dimensionalvolume.
 7. The apparatus of claim 1, wherein the prostate assessor is tomodel the prostate gland with a digital twin.
 8. A computer-readablestorage medium including instructions which, when executed, cause atleast one processor to at least: evaluate a volume and density of aprostate gland in an image of a patient to determine a prostate-specificantigen level for the prostate gland; analyze a lesion on the prostategland in the image; and generate an assessment of prostate gland healthbased on the prostate-specific antigen level and the analysis of thelesion.
 9. The computer-readable storage medium of claim 8, wherein theinstructions, when executed, cause the at least one processor to analyzethe lesion using a plurality of scores to form a global score toclassify the lesion.
 10. The computer-readable storage medium of claim8, wherein the instructions, when executed, cause the at least oneprocessor to evaluate the volume and density of the prostate gland bysegmenting and modeling the image using a deep learning network model.11. The computer-readable storage medium of claim 8, wherein theinstructions, when executed, cause the at least one processor to analyzethe lesion by segmenting and modeling the image using a deep learningnetwork model.
 12. The computer-readable storage medium of claim 8,wherein the instructions, when executed, cause the at least oneprocessor to generate an ellipse to be overlaid on the lesion, theoverlay of the ellipse to trigger automatic selection of one or moresectors underneath the ellipse in the image.
 13. The computer-readablestorage medium of claim 8, wherein the image includes athree-dimensional volume.
 14. The computer-readable storage medium ofclaim 8, wherein the instructions, when executed, cause the at least oneprocessor to model the prostate gland with a digital twin.
 15. A methodfor computer-aided prostate condition diagnosis, the method comprising:evaluating, with at least one processor, a volume and density of aprostate gland in an image of a patient to determine a prostate-specificantigen level for the prostate gland; analyzing, with the at least oneprocessor, a lesion on the prostate gland in the image; and generating,with the at least one processor, an assessment of prostate gland healthbased on the prostate-specific antigen level and the analysis of thelesion.
 16. The method of claim 15, wherein analyzing the lesionincludes analyzing the legion using a plurality of scores to form aglobal score to classify the lesion.
 17. The method of claim 15, whereinevaluating the volume and density of the prostate gland includesevaluating the volume and density of the prostate gland by segmentingand modeling the image using a deep learning network model.
 18. Themethod of claim 15, wherein analyzing the lesion includes analyzing thelesion by segmenting and modeling the image using a deep learningnetwork model.
 19. The method of claim 15, further including generatingan ellipse to be overlaid on the lesion, the overlay of the ellipse totrigger automatic selection of one or more sectors underneath theellipse in the image.
 20. The method of claim 15, wherein the imageincludes a three-dimensional volume.